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Objectives 

The  wavelet  transform  has  emerged  as  an  exciting  new  tool  for  statistical  signal  processing,  The 
wavelet  domain  provides  a  natural  setting  for  many  applications  involving  reai-world  signals  and 
images,  especially  those  rich  in  singularities  (edges,  ridges,  and  other  transients). 

This  research  aims  to  extend  the  wavelet  transform  and  modeling  and  processing  algorithms  to 
handle  multidimensional  signals  that  are  smooth  save  for  singularities  along  lower-dimensional 
manifolds.  The  basic  building  blocks  are  (1 )  a  new  hypercomplex  wavelet  transform  (HWT)  that 
generalizes  the  complex  wavelet  transform  to  higher  dimensions  using  a  multidimensional 
Hilbert  transform  and  (2)  adaptive  dyadic  partitions  based  on  wedgelets ,  which  are  atomic  edge 
segments.  From  the  HWT,  we  will  develop  new  classes  of  nearly  shift-invariant  wavelet  frames 
that  are  oriented  along  lower-dimensional  subspaces.  Finally,  we  will  construct  statistical  models 
for  these  new  transforms  and  partitions  that  extend  the  hidden  Markov  tree  (HMT)  model 
developed  under  previous  AFOSR  support  to  accurately,  realistically,  and  efficiently  represent 
singularity  structure. 


Accomplishments 

Efficient  image  geometry  encoding  using  wavelets  and  wedgelets 

We  developed  a  prototype  image  compression  algorithm  based  on  marrying  wavelets  and 
wedgelets  that  encodes  the  image  geometry  description  and  the  residual  texture  information 
jointly  in  a  rate/distortion  optimal  fashion.  The  algorithm  outperforms  the  new  JPEG2000  by  a 
fairly  significant  margin  (up  to  l-2dB  in  PSNR  for  "edgy”  images;  see  Figure  I).  Indeed,  we 
have  proved  that  for  a  simple  class  of  "piecewise  smooth”  cartoon  images  consisting  of  smooth 
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(C2)  regions  delineated  by  smooth  (C2)  edges,  that  this  algorithms  is  near-optimal  in  a 
rate/distortion  sense.  That  is,  the  mean-squared  error  D  of  image  approximation  decays  with  the 
number  of  bits  R  as  D(R)  -  C  (AVlog  N)'~.  Finally,  since  the  compressed  bit  stream  contains 
explicit  geometry  information  regarding  edges,  it  could  prove  very'  useful  for  efficient  image 
classification  and  data  base  searching.  This  new  framework  demands  further  investigation,  in 
particular  for  adaptive  compression  and  joint  compression/classification  of  geometric  targets. 

Dual-tree  quaternion  wavelet  transform  (QWT) 

Encoding  and  estimation  of  the  relative  locations  of  image  features  plays  an  important  role  in 
image  processing  applications,  ranging  from  feature  detection  and  target  recognition  to  image 
compression.  For  example,  in  edge  detection,  the  goal  is  to  locate  the  boundary  of  an  object  in  an 
image.  In  image  denoising  or  compression,  state-of-the-art  techniques  achieve  significant 
performance  improvements  by  exploiting  information  on  the  relative  locations  of  large  transform 
coefficients. 

An  efficient  way  to  compute  and  represent  relative  location  information  in  one-dimensional  (1- 
D)  signals  is  through  the  phase  of  the  Fourier  transform.  The  Fourier  shift  theorem  provides  a 
simple  linear  relationship  between  the  Fourier  phase  and  the  signal  shift.  When  only  a  local 
region  of  the  signal  is  of  interest,  the  short-time  Fourier  transform  (STFT)  provides  a  local 
Fourier  phase  for  each  windowed  portion  of  the  signal. 

The  discrete  wavelet  transform  (DWT)  is  more  natural  than  the  STFT  for  signals  containing 
isolated  singularities,  such  as  piecewise  smooth  functions.  The  locality  of  the  wavelet  basis 
functions  leads  to  a  sparse  representation  of  such  signals  that  compacts  the  signal  energy  into  a 
small  number  of  coefficients.  Wavelet  coefficient  sparsity  is  the  key  enabler  of  algorithms  such 
as  wavelet-based  denoising  by  shrinkage. 

While  wavelets  have  been  successfully  applied  to  many  signal  and  image  processing 
applications,  one  of  the  major  problems  with  conventional  real-valued  wavelets  is  their  lack  of 
shift-invariance.  A  small  shift  of  the  signal  results  in  significant  fluctuations  of  wavelet 
coefficient  energy,  making  it  difficult  to  extract  or  model  signal  information  from  the  coefficient 
values.  There  is  also  no  notion  of  phase  to  encode  signal  location  information  as  in  the  Fourier 
case.  Complex  wavelet  transforms  (CWTs),  such  as  the  dual-tree  CWT.  can  remedy  these 
problems.  In  1-D.  the  dual-tree  CWT  uses  complex  wavelet  basis  functions  whose  real  and 
imaginary  parts  form  a  1-D  Hilbert  transform  pair.  The  1-D  dual-tree  CWT  is  slightly  (two 
times)  redundant,  but  the  magnitudes  of  its  coefficients  are  almost  shift  invariant.  The  CWT 
phase  also  contains  information  on  the  locations  of  signal  singularities.  An  important  precursor 
of  the  dual-tree  approach  is  the  complex  Daubechies  wavelet  transform,  which  is  orthonormal 
but  not  shift-invariant. 

In  2-D  images,  a  single  complex  phase  is  not  enough  to  encode  image  shifts.  For  example,  in 
conventional  2-D  Fourier  analysis,  as  the  underlying  images  translates,  the  phase  value  is  a  linear 
combination  of  the  horizontal  and  vertical  image  shifts,  and  inference  of  the  shifts  from  a  single 
phase  vaiue  is  impossible.  The  same  phase  ambiguity  problem  exists  in  the  2-D  CWT,  which 
measures  phase  shifts  only  in  directions  perpendicular  to  its  wavelet  orientations. 
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To  remedy  this  problem,  we  married  the  quaternion  Fourier  transform  (QFT)  with  the  dual-tree 
CWT  to  develop  a  new  quaternion  wavelet  transform  (QWT)  for  images  and  other  2-D  data.  The 
QWT  leverages  the  notions  of  2-D  Hilbert  transform,  2-D  analytic  signal,  and  quaternion 
algebra.  The  2-D  Hilbert  transforms  of  a  usual  2-D  DWT  tensor  product  wavelet  (that  is,  the  1-D 
Hilbert  transform  of  the  wavelet  along  either  or  both  of  the  horizontal  and  vertical  directions) 
plus  the  wavelet  itself  constitute  the  four  components  of  each  quaternion  wavelet  (see  Figure  2). 
The  QWT  frame,  which  can  be  efficiently  generated  from  a  dual-tree  filter  bank,  is  a  4x 
redundant  tight  frame  that  is  stably  invertible. 

Image  localization  via  QWT 

The  QWT  has  a  quaternion  magnitude-phase  representation  that  encodes  image  shifts  in  an 
absolute  (xy)-coordinate  system  and  thus  provides  a  theoretical  framework  for  analyzing  the 
phase  behavior  of  2-D  image  shifts.  We  have  conducted  a  thorough  analysis  of  the  QWT  phase 
around  edge  regions.  We  have  also  developed  an  efficient  multiscale  flow/motion  estimation 
algorithm  for  image  registration  based  on  the  QWT  phase.  Figure  3  illustrates  our  results. 

Disparity  estimation  via  QWT 

The  QWT  has  a  quaternion  magnitude-phase  representation  that  encodes  image  shifts  in  an 
absolute  (x.yj-coordinate  system  and  thus  provides  a  theoretical  framework  for  analyzing  the 
phase  behavior  of  2-D  image  shifts.  A  thorough  analysis  of  the  QWT  phase  around  edge  regions 
has  enabled  us  to  develop  an  efficient  multiscale  flow/motion  estimation  algorithm  for  image 
registration  based  on  the  QWT  phase. 

The  algorithm  estimates  the  local  disparities  between  a  target  image  A(x,y)  and  a  reference 
image  B(x,y)  by  determining  the  local  translations  needed  to  align  different  regions  in  two 
images,  that  is,  the  amount  of  2-D  translation  required  to  move  a  local  region  of  a  target  image 
centered  at  pixel  (xO,  yO)  to  align  with  the  region  in  a  reference  image  centered  at  the  same 
location  (xO,  yO).  This  problem  figures  prominently  in  a  range  of  image  processing  and  computer 
vision  tasks,  including  video  processing  to  estimate  motion  between  successive  frames,  super¬ 
resolution,  etc.  The  algorithm  is  competitive  with  much  more  complicated  algorithms  such  as  the 
Gradient  Correlation  and  Exhaustive  Search  algorithms;  it  offers  sub-pixel  accuracy,  linear 
computational  complexity,  and  a  multiscale  structure  that  is  amenable  to  regularization  with 
noisy  data.  Figure  4  illustrates  our  results. 

Hypercomplex  wavelets  in  3-D  and  beyond 

We  have  also  extended  the  QWT  to  handle  multidimensional  signals  that  are  smooth  save  for 
singularities  along  lower-dimensional  manifolds.  Using  a  hypercomplex  wavelet  transform 
(HWT)  as  a  building  block,  we  have  constructed  new  classes  of  nearly  shift  invariant  w-avelet 
frames  that  are  oriented  along  lower-dimensional  subspaces.  The  HWT  can  be  computed 
conveniently  and  efficiently  using  a  1-D  dual-tree  complex  wavelet  transform  along  each  signal 
axis.  A  series  of  denoising  experiments  demonstrates  that  the  HWT  significantly  outperforms 
both  standard  tensor  product  wavelets  and  complex  wavelets  when  the  signals  of  interest  sport  a 


3 


local  manifold  structure.  See  Figure  5  for  an  example  of  three  hypercomplex  wavelets  in  3-D. 
Note  their  strong  1-D  orientation,  which  could  be  useful  for  analyzing  aircraft  trajectories 
through  3-D  space. 

Compressive  sensing 

Compressive  Sensing  is  an  emerging  field  based  on  the  revelation  that  a  small  number  of  linear 
projections  of  a  compressible  signal  contain  enough  information  for  reconstruction  and 
processing.  It  has  many  promising  implications  and  enables  the  design  of  new  kinds  of  data 
acquisition  systems,  imaging  systems,  and  cameras.  We  have  started  to  explore  a  new  camera 
architecture  that  employs  a  digital  micromirror  array  to  perform  optical  calculations  of  linear 
projections  of  an  image  onto  pseudorandom  binary  patterns  (see  Figure  6).  Its  hallmarks  include 
the  ability  to  obtain  an  image  with  a  single  detection  element  while  sampling  the  image  fewer 
limes  than  the  number  of  pixels  (sub-Nyquist  measuring).  Other  attractive  properties  include  its 
universality,  robustness,  scalability,  progressivity,  and  computational  asymmetry,  The  most 
intriguing  feature  of  the  system  is  that,  since  it  relies  on  a  single  photon  detector,  it  can  be 
adapted  to  image  at  wavelengths  that  are  currently  impossible  with  conventional  CCD  and 
CMOS  imagers,  Examples  of  compressive  image  acquisition  are  given  in  Figure  7. 
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Control,  and  Computing,  Allerton,  IL,  September  2005. 

S.  Sarvotham,  D.  Baron,  and  R.G.  Baraniuk,  “Variable-Rate  Coding  with  Feedback  for  Universal 
Communication  Systems,”  Allerton  Conference  on  Communication,  Control,  and  Computing, 
Allerton.  IL,  September  2005. 

M.  Wakin,  D.  L.  Donoho,  H.  Choi,  and  R.  G.  Baraniuk.  “The  Multiscale  Structure  of  Non- 
Differentiable  Image  Manifolds,"  Wavelets  XI  in  SP1E  International  Symposium  on  Optical 
Science  and  Technology,  San  Diego,  August  2005. 

W.  Chan,  H.  Choi,  and  R.  G.  Baraniuk,  “Coherent  Image  Processing  using  Quaternion 
Wavelets,"  Wavelets  XI  in  SPIE  Internationa!  Symposium  on  Optical  Science  and  Technology, 
San  Diego,  August  2005. 

R.  Wagner.  V.  Delouille,  H.  Choi,  and  R.  G.  Baraniuk,  “Distributed  Wavelet  Analysis  for 
Irregular  Sensor  Network  Grids,"  IEEE  Workshop  on  Statistical  Signal  Processing  -  SSP'05, 
Bordeaux,  France,  July  2005. 

S.  Palchaudhuri.  R.  Kumar,  R.  G.  Baraniuk,  and  D.  B.  Johnson,  “Design  of  Adaptive  Overlays 
for  Multi-scale  Communication  in  Sensor  Networks,"  IEEE  International  Conference  on 
Distributed  Computing  in  Sensor  Systems,  Columbus,  OH,  June  2005. 

S,  Sarvotham,  D.  Baron,  R.  G.  Baraniuk,  “Non-Asymptotic  Performance  of  Symmetric  Slepian- 
Wolf  Coding,"  Conference  on  Information  Sciences  and  Systems  (CISS).  Baltimore,  March 
2005. 

M.  Wakin,  D.  Donoho,  H.  Choi,  and  R.  G.  Baraniuk,  “Multiscale  Navigation  on  Non- 
Differentiable  Manifolds,”  special  session  on  Higher-Dimensional  Geometry  in  Signal 
Processing.  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal  Processing  - 
ICASSP’05,  Philadelphia,  March  2005. 

R.  Wagner,  S.  Sarvotham,  and  R.  G.  Baraniuk,  “A  Multiscale  Data  Representation  for 
Distributed  Sensor  Networks,”  IEEE  International  Conference  on  Acoustics,  Speech  and  Signal 
Processing  — ICASSP'05,  Philadelphia,  March  2005. 

H.  Choi  and  R.  G.  Baraniuk,  “Multiscale  Manifold  Representation  and  Modeling,”  IEEE  Inter¬ 
national  Conference  on  Acoustics,  Speech  and  Signal  Processing  -  ICASSP'05.  Philadelphia. 
March  2005. 
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N.  Ahmed  and  R.  G.  Baraniuk,  “Delay-limited  Throughput  Maximization  for  Fading  Channels 
using  Rate  and  Power  Control,”  IEEE  Globecom,  Symposium  on  Wireless  Communications, 
Networks  and  Systems,  Dallas,  December  2004. 

N.  Ahmed  and  R.  G.  Baraniuk,  "Throughput  Maximization  for  ARQ-like  Systems  in  Fading 
Channels  with  Coding  and  Queuing  Delay  Constraints,"  38th  Asilomar  Conference  on  Signals, 
Systems,  and  Computers,  Pacific  Grove.  CA,  November,  2004. 

D.  Baron,  A.  Khojastepour,  and  R.  G.  Baraniuk,  “How  Quickly  Can  We  Approach  Capacity  for 
the  Gaussian  Channel?”  38th  Asilomar  Conference  on  Signals,  Systems,  and  Computers,  Pacific 
Grove.  CA.  November,  2004. 

W.  Mantzel, !  1.  Choi,  and  R.  G.  Baraniuk,  "Distributed  Camera  Network  Localization,”  38th 
Asilomar  Conference  on  Signals,  Systems,  and  Computers,  Pacific  Grove,  CA,  November. 

2004. 

D.  Baron,  A.  Khojastepour,  and  R.  G.  Baraniuk,  “Excess  rates  of  non-asymptotic  Slepian-Wolf 
coding,"  Forty-Second  Annual  Alierlon  Conference  on  Communication,  Control,  and 
Computing,  Allerton.  IL,  September  2004. 

W.  Chan,  1 L  Choi,  and  R.  G.  Baraniuk,  “Quaternion  Wavelets  for  Image  Analysis  and 
Processing,”  IEEE  International  Conference  on  Image  Processing  -  ICI P-2004,  Singapore, 
October  2004. 


Interactions/T  ransitions 

Presentations 

2006  Keynote  and  plenary'  presentations  at  the  5th  Annual  ASEE  Global  Colloquium  on 
Engineering  Education,  (Rio  de  Janeiro,  Brazil),  12th  IEEE  Digital  Signal  Processing  Workshop, 
EPFL  Bernoulli  Center  Workshop  on  Wavelets  and  Applications,  TED  -  Technology, 
Entertainment,  and  Design  plus  the  usual  presentations  at  IMA,  T1  Developers  Conference, 
Google,  Michigan  State,  Boston  U.,  Toledo,  LANL,  and  the  AMD  Global  Vision  Conference. 

2005  IMA,  Fashion  Institute  of  Technology,  CMU,  UC-Berkeley,  MSRI,  TI  Developers 
Conference,  ExxonMobil,  NL11  Conference,  USC  Annenberg  Center,  ConocoPhillips,  Texas 
Instruments,  MIT,  UCLA  IPAM 

Consulting 

None  relevant  to  this  project. 
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Transitions 


Technology  on  the  Rice  compressive  imager  has  been  transferred  to  Texas  Instruments  for 
possible  incorporation  into  next-generation  imaging  systems. 

Projects 

None  directly  relevant  this  project. 


New  Discoveries,  Inventions,  Patent  Disclosures 

Patents  Hied:  (I)  Method  and  Apparatus  for  Compressive  Imaging  Device,  (2)  Method  and 
Apparatus  for  Optical  Image  Compression,  (3)  Method  and  Apparatus  for  Reconstruction  of 
Data  from  Multiple  Sources,  and  (4)  Method  and  Apparatus  for  Compressive  Imaging  Device. 


Honors  and  Awards 

2007:  Richard  Baraniuk  received  Edutopia  Magazine’s  “Daring  Dozen"  Education  Innovators, 
MIT  Technology  Review  TRIO  Top  10  Emerging  Technology  for  the  Rice  Single-Pixel  Camera. 
Hershei  M.  Rich  Invention  Award  (Rice). 

2006:  Richard  Baraniuk  received  the  Tech  Museum  of  Innovation  Laureate  for  Connexions 
(cnx.org)  and  the  George  R.  Brown  Aw'ard  for  Superior  Teaching  at  Rice  (third  time)  in  2006, 

2004:  Richard  Baraniuk  was  elevated  to  the  Victor  E.  Cameron  Chaired  Professorship  in 
Electrical  and  Computer  Engineering  at  Rice. 
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Figure  1:  Comparison  of  state-of-the-art  SFQ  wavelet-based  image  coder 
versus  the  new  WSFQ  coder  based  on  wavelets  and  wedglets  (zoom). 
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Figure  2:  Quaternion  wavelets  and  scaling  function  in  2-D.  Each  wavelet  is 
90  degrees  phase  shifted  from  the  others  in  either 
the  horizontal,  vertical,  or  both  directions. 


1 1 


Figure  3:  Quaternion-based  image  flow  estimation.  Top:  Two  images  from  a  video  sequence  of 
a  spinning  cube.  Bottom:  Multiscale  image  flow  estimate  base  on  the  QWT  of  the  images. 

The  algorithm  is  both  efficient  and  accurate  and  applicable  to  a  wide  range  of  video  processing 

applications. 
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Figure  4:  Quaternion-based  image  flow  estimation.  Two  images  from  a  video  sequence  of  a 
beating  heart.  Multiscale  image  flow  estimate  base  on  the  QWT  of  the  images.  The  algorithm  is 
both  efficient  (linear  complexity  in  the  number  of  pixels)  and  accurate  (sub-pixel  accuracy)  and 
applicable  to  a  wide  range  of  image  and  video  processing  applications. 


Figure  5:  Quaternion  wavelets  and  seating  function  in  3-D.  Each  wavelet  is 
90  degrees  phase  shifted  from  the  others  in  either 
the  horizontal,  vertical ,  or  both  directions. 
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Figure  6:  Compressive  sensing,  “ single  pixel"  camera  architecture  that  optically  computes 
inner  products  between  the  scene  being  imaged  and  pseudorandom  patterns.  Back-end  signal 
processing  reconstructs  the  image  from  the  random  measurements. 


ideal 


lOx  sub-Nyquist 


50x  sub-Nyquist 


Figure  7:  Example  images  acquired  by  the  single-pixel  camera  reconstructed 
from  measurements  at  l  Ox  and  50x  below  the  Nyquist  rate. 
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